Segmenting and tracking of video objects has many applications in computer vision, and video-processing applications such as object based coding and video post-production. Current segmentation and tracking of video objects involve object segmentation, in the first video frame, followed by tracking the object in the following frames. The tracking depends on the segmentation method used to segment the first frame.
Current segmentation techniques are either region based, boundary-based or a combination of region and boundary techniques. Region based segmentation groups pixels into regions based on their proximity and homogeneity. The region-based segmentations produce grouping by split and merge until a certain criteria is satisfied. Region based algorithms tend to over segment the object and has difficulty with complex textured objects. Boundary based segmentation tracks the boundary of the object in, for example, it maps a curve on the object boundary. The boundary-based methods do not take into account the content of the object. These methods require good initial contour processing and have problems with separate regions wholly contained within the region boundary. Combinations of region and boundary segmentation techniques restrict region growing near an object boundary. In general, the region and boundary combination techniques produce good results but are time consuming. Current tracking algorithms highly depend on the segmentation method used in the first frame. With region based segmentation tracking consists of estimating the motion of the region in the following frames. If boundary or curve based segmentation is used, the tracking consists of finding the curve location in the following frames.
Current segmentation and tracking of video objects do not take advantage of the knowledge of the object type. These segmentation and tracking techniques have limited functionality and work only for a certain class of objects. Many of the current approaches are semi-automatic in that they require the user to identify the object of interest and to manually correct inaccuracies.